System and Method for Defining an Augmented Reality View in a Specific Location

ABSTRACT

This invention is a system and method for defining a location-specific augmented reality capability for use in portable devices having a camera. The system and method uses recent photographs or digital drawings of a particular location to help the user of the system or method position the portable device in a specific place. Once aligned, a digital scene is displayed to the user transposed over (and combined with) the camera view of the current, real-world environment at that location, creating an augmented reality experience for the user.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional PatentApplication No. 61/615,321 filed by the same inventors on Mar. 25, 2012.

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

JOINT RESEARCH AGREEMENT

Not applicable

BACKGROUND OF THE INVENTION

(1) Field of the Invention

This invention relates to computerized virtual reality capabilities inspecific locations. This invention further relates to using computergraphics processing and selective visual display systems. This inventionis specifically a system and method for defining a means to showaugmented reality scenes in specific locations.

This invention is intended to be employed for technical uses andsolutions when a specific location is required, for AR applications suchas gaming and social networking, as well as for educational andentertainment uses such as viewing a historical image or model in ahistorically accurate location.

(2) Description of Related Art Including Information Disclosed Under 37CFR 1.97 and 37 CFR 1.98

Augmented reality (AR) is a live, direct or indirect, view of aphysical, real-world environment whose elements are augmented bycomputer-generated sensory input such as sound, video, graphics or GPSdata. It is related to a more general concept called mediated reality,in which a view of reality is modified by a computer. As a result, thetechnology functions by enhancing one's current perception of reality.By contrast, virtual reality replaces the real world with a simulatedone.

Augmentation is conventionally in real-time and in semantic context withenvironmental elements, such as sports scores on TV during a match. Withthe help of advanced AR technology (e.g. adding computer vision andobject recognition) the information about the surrounding real world ofthe user becomes interactive and digitally manipulable. Artificialinformation about the environment and its objects can be overlaid on thereal world.

The AR field is divided into these major areas from an algorithmicstandpoint:

-   -   marker-based    -   positional-based    -   object/feature recognition

Marker-based augmented reality is based on the computer identifyingartificial markers in the real world (examples: QR codes, barcodes, orsimilar markers) and superimpose computer-generated images based onwhere the markers are located. This area requires significant imageprocessing tasks done by the computer.

Positional-based augmented reality is based on where you are located,where you are pointing to (as in heading), and where are the objects ofinterest are located relative to you. The computer then will superimposeimages on top of the real-world image gathered. The computer doesn'tneed to do much image processing (almost none at all) except forsuperimposing the generated image on top of the camera image.

Object/feature recognition is the process whereby the computer willrecognize real-world objects directly and thus the markers are no longerneeded, but it is still a topic that requires much research.

Traditional object/feature recognition and positional based processingis processing that is intensive and highly inaccurate. To some extentthis is also true of marker based systems. One problem associated withmarker tags is that they must be visible and easily recognized for thecamera to recognize and interpret.

BRIEF SUMMARY OF THE INVENTION

A system and method for defining a location-specific augmented realitycapability for use in portable devices having a camera. The method usesrecent photographs or digital drawings of a particular location to helpthe user of the system or method position the portable device in aspecific place. Once aligned, a digital scene is displayed to the usertransposed over (and combined with) the camera view of the current,real-world environment at that location.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates the user aligning the superimposed Digital ImageMarker on the Viewer Context, in this embodiment on a mobile device, ina Particular Location.

FIG. 1A illustrates the user indicating the device is properly alignedwith the Digital Image Marker.

FIG. 1B illustrates a screenshot of the Digital Image Markersuperimposed on the Viewer Context, in a Particular Location.

FIG. 2 illustrates the user in a Particular Location viewing the DigitalArtifact being superimposed with low opacity on the Viewer Context of amobile device.

FIG. 2A illustrates a screenshot of the Digital Artifact beingsuperimposed with low opacity on the Viewer Context of a mobile devicein a Particular Location.

FIG. 3 illustrates the user in a Particular Location viewing the DigitalArtifact being superimposed with high opacity on the Viewer Context of amobile device.

FIG. 3A illustrates a screenshot of the Digital Artifact beingsuperimposed with high opacity on the Viewer Context, in this embodimenton a mobile device, in a Particular Location.

FIG. 4 illustrates the user aligning the superimposed Digital ImageMarker on the Viewer Context, in this embodiment on a mobile device, ina different Particular Location.

FIG. 4A illustrates the user indicating the device is properly alignedwith the Digital Image Marker, in a different Particular Location.

FIG. 4B illustrates a screenshot of the Digital Image Markersuperimposed on the Viewer Context, in a different Particular Location.

FIG. 5 illustrates the user in a different Particular Location viewingthe Digital Artifact being superimposed with low opacity on the ViewerContext of a mobile device.

FIG. 5A illustrates a screenshot of the Digital Artifact beingsuperimposed with low opacity on the Viewer Context of a mobile devicein a different Particular Location.

FIG. 6 illustrates the user in a different Particular Location viewingthe Digital Artifact being superimposed with high opacity on the ViewerContext of a mobile device.

FIG. 6A illustrates a screenshot of the Digital Artifact beingsuperimposed with high opacity on the Viewer Context, in this embodimenton a mobile device, in a different Particular Location.

FIG. 7 illustrates the steps an end user of the invention would performto use the invention.

DETAILED DESCRIPTION OF THE INVENTION

The following description of the invention is merely exemplary in natureand is not intended to limit the invention or the application or uses ofthe invention.

This invention is a system and method designed to manage and displaydigital scenes or images related to a specific location and place, andmerged with the current, real-world environment at that specificlocation.

Referring to FIG. 1, the invention uses specially created images (i.e.Digital Marker Images) to position a Viewer Context in a ParticularLocation.

One embodiment of this invention uses the sequence of images presentedto a user through the camera function on a Smartphone or other portabledevice as the Viewer Context. Therefore, this embodiment of theinvention positions the user of the invention in a specific place,looking through their Smartphone or portable device in a specificdirection.

Referring to FIG. 1B, the invention allows the user of the invention toview the Marker Image as an overlay in the current Viewer Context.Referring to FIG. 1A, the invention allows the user of the invention toindicate that they are positioned properly.

Referring to FIG. 2, after the user indicates they are positionedproperly, the invention then will display a Digital Artifact to theuser. The Digital Artifact that is displayed is associated in somemanner with the Particular Location.

Referring to FIG. 2A, the invention allows the user to view the current,real-world environment simultaneously, or merged with, the DigitalArtifact.

Referring to FIGS. 3 and 3A, while the Digital Artifact is displayed,and as the user changes the Viewer Context (in this embodiment, anopacity control is manipulated), the Digital Artifact may be modified orrepositioned in the Viewer Context as well.

Referring to FIG. 7, the invention provides a sequence of steps. The enduser can navigate through those steps to:

-   -   Position their device in the proper location    -   Indicate that the device is positioned properly    -   View the Digital Artifact combined with the current Viewer        Context    -   Adjust the view to observe changes in the Digital Artifact

Definitions. Particular Location Definition:

A Particular Location is defined for the purposes of this invention tobe the combination of a specific place in the world, including aparticular azimuth and altitude, relative to the observer.

Viewer Context Definition:

A Viewer Context is defined for the purposes of this invention to be thedigital representation of an actual scene, presented on a portabledevice. A Viewer Context is dynamic, and responds rapidly to newinformation from the scene, as the device is moved.

The most common Viewer Context for this invention is the sequence ofimages presented to a user through the camera function on a Smartphoneor other portable device.

Content Definition:

Content for the purposes of this invention consists of two types ofdigital assets:

-   -   a) A Digital Marker Image that is used to help the user of the        invention position the Viewer Context in a Particular Location.    -   b) A Digital Artifact that is displayed in the Viewer Context        after a Particular Location has been acquired by the invention.        This artifact adds to the user's viewing experience while        looking at a Particular Location.

Digital Marker Images Definition:

As defined in this document, Digital Marker Images are used to help theuser position the Viewer Context in a Particular Location. The DigitalMarker Image is added to, or presented in the Viewer Context as asemi-transparent layer, superimposed on the Viewer Context (e.g., theview acquired in real-time from a camera on a supported device). TheDigital Marker Image is, in some manner, related to the Viewer Contextfor a Particular Location. The Digital Marker Image helps the user alignthe Viewer Context with a Particular Location.

Several types of Digital Marker Images are supported, including:

-   -   a) A digital picture of the Particular Location, taken recently.    -   b) A digitized 2 dimensional drawing that highlights unique        characteristics of the particular location.

Digital Marker Image characteristics:

Several types of Digital Marker Images are supported by the invention.Each marker must contain these characteristics or provide thesefeatures:

-   -   a) Contain unique features (i.e. unique attributes or shapes)        that relate to those observed in the scene presented in the        Viewer Context, and are readily recognizable by the user.    -   b) Are semi-transparent, so that the user can observe both the        Digital Marker Image and the Viewer Context.

Digital Artifact Definition:

Digital Artifacts are added to, or presented in the Viewer Context aftera Particular Location has been acquired. Digital Artifacts add to theuser's visual experience and present additional information to the userabout the Particular Location.

Several types of Digital Artifacts are supported by the invention,including:

-   -   a) A digital picture of the Particular Location, taken at some        point in the past.    -   b) A digitized 2 dimensional drawing associated with the        Particular Location.    -   c) A digitized 3 dimensional model associated with the        Particular Location.

Digital Artifact characteristics:

Several types of Digital Artifacts are supported by the invention.

Each Digital Artifact must contain these characteristics or providethese features:

-   -   a) Is related to the Particular Location, in some manner. In one        embodiment, the Digital Artifact may be a historical image or        model, representing what that Particular Location looked like in        the past. Or, in another embodiment, the Artifact can represent        what that location will look like in the future. In a further        embodiment, the Artifact would also be an artistic enhancement        that adds information to the context.    -   c) Allow some portion or portions of the Viewer Context to        remain visible. This is so that the user can still see some        portion of the Viewer Context while the Digital Artifact is        displayed.

Digital Artifacts may contain these characteristics or provide thesefeatures:

-   -   a) Contain attributes or shapes that augment those observed in        the Viewer Context, and add to or enhance the information        presented to the user.    -   b) Obscure portions of the Viewer Context with content from the        artifact.    -   c) Digital Artifacts can be either semi-transparent or        non-transparent.    -   d) Allow the system to rotate, resize or reposition the Artifact        in response to changes in the Viewer Context. Examples of        changes include moving closer to or further from the Particular        Location, or changing the altitude or azimuth of the Viewer        Context.

OBJECTS ADVANTAGES OF THE INVENTION

It is an object of the present invention to provide an improved markerfor AR applications.

It is another object of the present invention to provide a systemwhereby handheld and portable computing devices having cameras and imageprocessing technology such as cell phones and tablets can be used in ARenabled applications.

Another object of the invention is to enhance identification of specificlocations with limited computer processing requirements.

It is a further object of the invention to provide a method for definingan AR scene in a computer AR application such as an application to viewhistorical images in historically accurate locations.

Additional objects, advantages and novel features of the invention willbe set forth in part of the description which follows, and in part willbecome apparent to those skilled in the art upon examination of thefollowing specification.

A unique feature of this invention is the ability to use digital markerimages to help the user of the invention to position the device in aparticular location. Once that particular location is viewable to theuser of the invention in the viewer context, the invention will displaya specific digital artifact to the user.

What is claimed is:
 1. A system for defining an augmented realitycapability for a portable computer, said system comprising: a) aportable camera having the ability to show the current real-worldenvironment; b) a portable computer having the ability to show images,drawings, and models; c) a software program executed by said computerprocessor for managing the display of said images, drawings, and models;d) a set of controls whereby the user can interact with the computerprogram; e) an extensible collection of marker images containing aplurality of marker images that represent a specific view of aparticular location; and f) an extensible collection of digitalartifacts containing a plurality of digital artifacts that areassociated with a specific view of a particular location; wherein: a)the user of the system requests a particular marker image within the setof marker images; b) the user of the system aligns the digital markerimage and the current real-world environment as viewed through thecamera; c) the user of the system indicates the camera and the digitalmarker image are aligned; d) the system displays a digital artifactassociated with that particular location; e) the user of the systemviews the digital artifact and the current real-world environmentsimultaneously f) the user of the system manipulates the camera viewwith the provided controls; and g) the user of the system the adjusteddigital artifact and the current real-world environment simultaneously.2. The system for linking a location with a marker image in accordancewith claim 1, wherein said marker images is comprised at least one fromthe groups: a) A digital picture of a particular location, takenrecently; or b) A digitized 2 dimensional drawing that highlights uniquecharacteristics of the particular location.
 3. The system for linking alocation with a marker image in accordance with claim 1, wherein saidmarker images must have these characteristics: a) Contain uniquefeatures (i.e. unique attributes or shapes) that relate to thoseobserved in the scene presented in the Viewer Context, and are readilyrecognizable by the user; and b) Are semi-transparent, so that the usercan observe both the Digital Marker Image and the Viewer Context.
 4. Thesystem for linking a location with a digital artifact in accordance withclaim 1, wherein said digital artifacts must have these characteristics:a) can be added to, or presented in the viewer context after aparticular location has been acquired; b) add to the user's visualexperience and present additional information to the user about theparticular location.
 5. The system for linking a location with a digitalartifact in accordance with claim 1, wherein said digital artifacts iscomprised at least one from the groups: a) A digital picture of theparticular location, taken at some point in the past; b) A digitized 2dimensional drawing associated with the particular location; or c) Adigitized 3 dimensional model associated with the particular location.6. The system for linking a location with a digital artifact inaccordance with claim 1, wherein said digital artifacts must have thesecharacteristics: a) is related to the particular location, in somemanner; and b) allows some portion or portions of the viewer context toremain visible.
 7. The system for linking a location with a digitalartifact in accordance with claim 1, wherein said digital artifacts mayhave these characteristics: a) contain attributes or shapes that augmentthose observed in the viewer context, and add to or enhance theinformation presented to the user; b) obscure or partly obscure portionsof the viewer context with content from the artifact; or c) allow thesystem to rotate, resize, or reposition the artifact in response tochanges in the Viewer Context.